The present invention relates to a texture description method for an image, and more particularly, to a method of describing image texture in the frequency domain, in which image signals are converted into those in a frequency domain of the Polar coordinate system to extract texture features. The present invention also relates to a method of texture-based retrieval of images indexed by the texture description method.
The texture information of an image is one of the most important visual characteristics of the image and thus, has been studied together with the color information for a substantial period of time. This texture information of the image is usually used as an important low-level visual descriptor in content-based indexing and in abstracting an image or video data. Also, image texture is very important information used for retrieval of a special picture in an electronic album or content-based retrieval in tiles or textiles database.
Until now, feature values have generally been computed in the time domain or in the frequency domain to extract a texture feature of the image. More particularly, the method of extracting the texture features in the frequency domain was known to be suitable for describing image texture information of various types. Extracting texture features in the frequency domain can be done in the Cartesian or the Polar coordinate system.
Conventionally, the Cartesian coordinate system has been widely used in extracting a texture feature in the frequency domain.
A paper entitled xe2x80x9cTexture Features For Browsing And Retrieval Of Image Dataxe2x80x9d, written by B. S. Manjunath and W. Y. Ma is published in xe2x80x9cIEEE Transaction on Pattern Analysis and Machine Intelligencexe2x80x9d, vol.18, no.8, in August of 1996, addresses a method of dividing the frequency domain of the Cartesian coordinate system based on HVS (Human Visual System) filtering of an image in the respective channels by Gabor filters, and then extracting the average and the standard deviation from the respective channels as texture features of the image.
However, the method of describing image texture is not suitable in the frequency domain of the Cartesian coordinate system for the HVS and leads to poor performance in relevant texture images.
To solve the problem of the image texture description method in frequency domain of the Cartesian coordinate system, a paper on image texture description method in frequency domain of the Polar coordinate system was published, in which the texture information in the frequency domain is computed in the Cartesian coordinate system.
In the paper entitled xe2x80x9cRotation-invariant Texture Classification using a complete Space Frequency Modelxe2x80x9d, written by B. S. Manjunath and Geoge M. Haley and published in xe2x80x9cIEEE Transaction on Pattern Analysis and Machine Intelligencexe2x80x9d, vol. 8, no.2, in February of 1999, a method of dividing a frequency space of the Polar coordinate system based on HVS (Hunan Visual System), then extracting 9 feature values using a Gabor filter designed to be suitable for respective channels, and describing the image texture using the extracted feature values of all channels was disclosed.
However, in this method, the same design of a set of Gabor filters is used for extracting different kinds of texture features in the frequency domain.
The disclosed embodiments of the present invention provide a texture description method in a frequency domain, suitable for HVS, in which image texture features are computed and indexed in a frequency domain.
In accordance with a further embodiment of the present invention, a texture-based retrieval method by using texture features computed in the frequency domain of the Polar coordinate system is provided, in which similar images in different variations, such as different rotations or scales or pixel intensity, are retrieved by comparing a query texture descriptor with a data texture descriptor generated by the texture description method and taking into account such variations thereof.
Also provided is a texture description method in the frequency domain of the Polar coordinate system that includes a first step of generating a frequency layout by partitioning said frequency domain into a set of feature channels; a second step of extracting texture feature values of said image from said respective feature channels; and a third step of constituting a texture descriptor in a vector form by using said texture feature values extracted from said respective feature channels in said frequency layout.
It is preferable that said first step is of generating said frequency layout on the basis of the HVS (Human Visual System), and that said frequency domain in said first step is that of the Cartesian coordinate system or the Polar coordinate system.
It is also preferable that said first step includes a sub-step of generating different frequency layouts for different types of texture features, that is, each texture feature type for its respective frequency layout.
It is further preferable that said first step comprises a sub-step of assigning significance or priority to the respective channels.
Also, it is preferable that said second step include a first sub-step of Radon-transforming said image; a second sub-step of Fourier-transforming said Radon-transformed image; and a third sub-step of extracting said texture feature values of said Fourier-transformed image from said respective feature channels.
It is further preferable that said third sub-step is of extracting at least energy deviation values and/or energy values in said respective feature channels.
Here, it is preferable that a frequency layout for obtaining said energy values and a frequency layout for obtaining said energy deviation value is separately prepared for extracting different types of an image texture, and that said frequency layout for obtaining said energy values partitions said frequency domain at intervals of 2l(0xe2x89xa6l less than log2(N/2)xe2x88x921) octave in a radial direction and at intervals of xe2x80x98180/dividing resolutionxe2x80x99 in an angular direction. The frequency layout for obtaining said energy deviation values partitions said frequency domain at the same intervals in a radial direction and at intervals of xe2x80x98180/dividing resolutionxe2x80x99 in an angular direction.
It is preferable that said third step is of finding out a rotational reference axis of said image by using said image texture information, rotating said frequency layout with reference to said rotational reference axis, and then extracting said image texture descriptor of said image. Here, the rotational reference axis is set to be an axis in a radial direction, in which one of energy, entropy, and a periodical component is most distributed by Radon-transforming said image.
Preferably, the third step is of Fourier-transforming said image to find out a radial reference point, normalizing said Fourier-transformed image with reference to said reference point, and then describing said texture descriptor by using said normalized values of said Fourier-transformed image. Here, the radial reference point is set by determining an arc in which one of energy, entropy, and a periodical component of said Fourier-transformed image apart at the same distance from the origin in said frequency domain is most distributed, and then setting a radius of said founded arc as said radial reference point.
It is preferable that the method of describing image texture in a frequency domain according to the present invention further comprise a fourth step of extracting intensity information of said image to add said intensity information to said texture descriptor.
Also, according to the present invention, a computer readable recording media recording a program for realizing a texture description method in a frequency domain is provided. The program performs a first step of generating a frequency layout by partitioning said frequency domain into a set of feature channels; a second step of extracting texture feature values of said image by Radon-transforming said image in said respective feature channels, Fourier transforming said Radon-transformed image, and extracting texture feature values of said Fourier-transformed image from respective feature channels; and a third step of constituting a texture descriptor of said image in vector form by using said texture feature values extracted from said respective feature channels.
Also, according to the present invention, a method of populating a database with texture descriptors of images is provided. The method includes a first step of generating a frequency layout by partitioning the frequency domain into a set of feature channels; a second step of extracting texture feature values of said images in said respective feature channels; a third step of constituting texture descriptors of said images in vector forms by using said texture feature values extracted in said respective feature channels of said frequency layout; and a fourth step of indexing said respective texture descriptors of said images into said database. The first step comprises a first sub-step of generating the respective frequency layouts for texture feature types by partitioning the frequency domain into the respective sets of feature channels; and a second sub-step of extracting said texture feature values of each type for said images in said feature channels of said respective frequency layouts.
It is preferable that said second sub step include a first step of extracting energy values of a texture feature type for said images in said feature channels of the corresponding frequency layout for said energy feature type; and a second step of extracting energy deviation values of a texture feature type for said images in said feature channels of the corresponding frequency layout for said energy deviation feature type.
Also, it is preferable that said third step include a first sub-step of constituting texture descriptors of said images with said energy values and energy deviation values in a vector form; and a second sub-step of adding the mean and standard deviation values of said images into each of said texture descriptors of said images.
More preferably, the second step includes extracting energy values and energy deviation values as texture features for said images in said feature channels of said frequency layout.
Still more preferably, the first sub-step includes a step of generating, based on the HVS, more than one frequency layout for which each frequency layout is used for extracting feature values of each feature type; and a step of assigning significance or priority to respective channels of said frequency layouts.
Also, it is preferable that the second sub-step include a step of Radon-transforming the inputted images; a step of Fourier-transforming said Radon-transformed image; and a step of extracting feature values from said Fourier-transformed image in said respective feature channels of said frequency layout. The step of extracting feature values from said Fourier-transformed image is of extracting at least energy values or energy deviation values from said respective feature channels of said frequency layout.
Also, a method of retrieving relevant texture images in a database similar to a query image according to the present invention is provided. The method includes a first step of generating a frequency layout by partitioning frequency domain into a set of feature channels for feature extraction of an input query image; a second step of extracting a query texture descriptor of said query image when said query image is inputted; a third step of reading a texture descriptor from said database; a fourth step of measuring a distance between said texture descriptor of said input texture image and said texture descriptor read from said database; a fifth step of measuring distances for said texture descriptor of said input image to all or at least parts of the texture descriptors in said database; and a sixth step of ordering the similarity for the said texture descriptor to said texture descriptors in said database using said measured distances.
It is preferable that when rotation-invariant matching of said image is considered, said fifth step include a first sub-step of measuring distances between a texture descriptor taken from said database and said query texture descriptor by shifting feature values of said query texture descriptor in angular directions into the corresponding positions where the shifted feature values are supposed to be extracted when said query image rotates; a second sub-step of measuring the distances between said texture descriptor of said input texture image to said texture descriptor stored in said database for all rotation angles; and a third sub-step of determining as said distance the minimum distance between aid texture descriptor of said input texture image and said texture descriptor stored in said database for all rotation angles.
Also, it is preferable that when scale-invariant matching of said image is considered, said fifth step include a first sub-step of forming at least one zoom-in image and/or zoom-out image from said query image and extracting said query texture descriptors of zoom-in and/or zoom-out images of said query image; a second sub-step of measuring the distances between said query texture descriptors of zoom-in and/or zoom-out query images and said data texture descriptor in said database; and a third sub-step of determining as the distance the minimum distance of said distances between said texture descriptor in said database and said texture descriptor of said query texture descriptors at different scale values. Here, it is preferable that said query texture descriptor and said texture descriptor in said database include a rotational reference axis, a radial reference point, and mean and stand deviation of texture image intensities, respectively.
It is further preferable that when rotation-invariant of said query texture descriptor is considered, said fifth step is of aligning said texture descriptor of said query image and said texture descriptor in said database with reference to given rotation angles.
Also, it is preferable that said rotational reference axes are set to be radial axes in which one of an energy, an entropy, and a periodical component is most distributed in Fourier transform of said Radon-transformed images.
Preferably, when intensity-invariant matching of said query texture descriptor is considered, said fifth step is of excluding mean values from said query texture descriptor and said texture descriptor in said database and measuring a distance between said two texture descriptors.
More preferably, when scale-invariant matching of said query texture image is considered, said fifth step comprises a first sub-step of merging said feature values of the adjacent channels in radial directions for said two texture descriptors to be compared or shifting feature values of said two texture descriptors into radial directions according to a radial reference point; and a second sub-step of measuring a distance between said two texture descriptors with feature values merged in adjacent feature channels or with feature values shifted into adjacent feature channels.
Here, said radial reference point is preferably set by determining an arc in which energy or entropy or periodical components of said Fourier-transformed image apart at the same distance from the origin in said frequency domain are most distributed and setting a radius of said determined arc as said radial reference point.
When scale-invariant and rotation-invariant matching is considered simultaneously, said fifth step includes a first sub-step of merging said feature values of the adjacent channels in radial directions for said two texture descriptors to be compared or shifting feature values of said two texture descriptors into radial directions with reference to a radial reference point; a second sub-step of shifting feature values of said two texture descriptors in angular directions into the corresponding positions where the shifted feature values are supposed to extracted with reference to a rotation point; and a third sub-step of measuring a distance between said two texture descriptors with feature values of adjacent feature channels merged in radial directions and then shifted in angular directions.
A computer readable recording media recording a program retrieving a data image similar to any query image in a computer according to the present invention is provided. The program performs the following steps: a first step of generating a frequency layout by partitioning the frequency domain into a set of feature channels; a second step of, when images to be stored in a database is given, extracting texture feature values of said data image in said respective feature channels, and then extracting and storing a data texture descriptor of said data image by using said extracted texture feature values; a third step of, when said query image is inputted, extracting texture feature values of said query image in said respective feature channels, and extracting a query texture descriptor of said query image by using said extracted texture feature values; a fourth step of matching said data texture descriptor with said query texture descriptor and measuring a distance between two texture descriptors; and a fifth step of determining a similarity between said two images by means of said distance between said two texture descriptors.
Also, a texture-based retrieval method of a data image similar to a query image in a frequency domain according to the present invention is provided. The method includes a first step of extracting and storing a texture descriptor including texture feature values and the rotation information of images to be stored in a database; a second step of extracting a query texture descriptor including texture feature values and the rotation information of said query image when said query image is inputted; a third step of aligning the rotating angle between said data texture descriptor and said query texture descriptor according to said rotation information of said two texture descriptors; a fourth step of matching said two texture descriptors and measuring a distance between said two texture descriptors with rotation angles aligned between said two texture descriptors; and a fifth step of determining a similarity between said two images by means of said distance between said two texture descriptors.
It is preferable that said step of extracting said texture descriptor in said first and second steps include a first sub-step of generating a frequency layout by partitioning the frequency domain into a set of feature channels so as to extract respective feature value; a second sub-step of extracting texture feature values of said images in said respective divided frequency domains; and a third sub-step of constituting a texture descriptor of said image in a vector form by using said feature values extracted in said respective frequency channels of said frequency layout.
It is more preferable that said step of extracting said rotation information of said images in said first and second steps include a first sub-step of finding out a direction in which energy is much distributed in the Fourier transform of said inputted image; a second sub-step of generating a frequency layout by using said direction as a reference axis; and a third sub-step of adding said rotation information of said frequency layout to said texture descriptor of said image.
It is still more preferable that said first sub-step in said step of extracting said texture descriptor includes a step of generating at least one frequency layout in consideration of HVS; and a step of giving significance or priority to respective feature channels of said frequency layouts.
Preferably, said second sub-step in said step of extracting said texture descriptor includes a step of Radon-transforming said inputted image; a step of Fourier-transforming said Radon-transformed image; and a step of extracting said texture feature values from said Fourier-transformed image with respect to said respective frequency layout, and it is preferable that the step of extracting texture feature values from said Fourier-transformed image is of extracting at least energy values or energy deviation values in said respective feature channels.
A computer readable recording media recording a program retrieving a data image similar to a query image in a computer according to the present invention is provided. The program performs a first step of generating a frequency layout by partitioning a frequency domain into a set of feature channels; a second step of generating and storing a data texture descriptor by extracting texture feature values and the rotation information of said data image from said respective feature channels when an image to be stored in a database is given; a third step of generating a query texture descriptor by extracting texture feature values and the rotation information of said query image from said respective feature channels when said query image is inputted; a fourth step of aligning the rotating angles between said two data texture descriptors by using said rotation information of said data texture descriptor and said rotation information of said query texture descriptor; a fifth step of matching said two texture descriptors and measuring a distance between said two texture descriptors with said rotating angles aligned between said two texture descriptors; and a sixth step of determining a similarity between said two images by means of said distance between said two texture descriptors.